Image sensor with photoelectric conversion units arranged in different directions

ABSTRACT

An imaging device includes: a first image sensor comprising first pixels that receive incident light, and that include a first and second photoelectric conversion units that are arranged in a first direction; and a second image sensor including second pixels that receive light that has passed through the first image sensor, and that include a third and fourth photoelectric conversion units that are arranged in a second direction that is different from the first direction.

This is a Continuation of application Ser. No. 16/672,571 filed Nov. 4,2019, which is a Continuation of application Ser. No. 15/556,011 filedSep. 6, 2017, which is a National Stage Entry Application ofPCT/JP2016/059233 filed Mar. 23, 2016, which in turn claims priority toJapanese Application No. 2015-071017 filed Mar. 31, 2015. The entiredisclosures of the prior applications are hereby incorporated byreference herein.

TECHNICAL FIELD

The present invention relates to an imaging device.

BACKGROUND ART

A digital camera that is equipped with an image sensor, with pixels eachof which has a micro lens and first and second photoelectric conversionunits are arranged two dimensionally on the image sensor, is per seknown (refer to Patent Document #1). Along with performing phasedifference method focus detection according to first and secondphotoelectrically converted signals from the first and secondphotoelectric conversion units, this digital camera also generates animage by adding together the first and second photoelectricallyconverted signals for each pixel. In phase difference focus detection,for making such focus detection possible on the basis of a pair ofimages that are formed by the pupil split in two mutually differentdirections, for example a pair of images that are pupil-divided in thevertical direction and in the horizontal direction, accordingly theimage sensor has first pixels in which the first and secondphotoelectric conversion units are arranged in series in the horizontaldirection, and second pixels in which the first and second photoelectricconversion units are arranged in series in the vertical direction. Dueto the provision of first and second pixels of these types, satisfactoryphase difference focus detection is enabled even for photographicsubject patterns that include many vertically striped patterns or manyhorizontally striped patterns.

CITATION LIST Patent Literature

Patent Document #1: Japanese Laid-Open Patent Publication 2007-65330.

SUMMARY OF INVENTION Technical Problem

There is the problem that the digital camera described above is notcapable of splitting the same image of the photographic subject indifferent directions by the split pupil.

Solution to Problem

According to the 1st aspect of the present invention, an imaging devicecomprises: a first image sensor comprising first pixels that receiveincident light, and that comprise first and second photoelectricconversion units that are arranged in a first direction; and a secondimage sensor comprising second pixels that receive light that has passedthrough the first image sensor, and that comprise third and fourthphotoelectric conversion units that are arranged in a second directionthat is different from the first direction.

According to the 2nd aspect of the present invention, it is preferablethat in the imaging device according to the 1st aspect, the first imagesensor photoelectrically converts light of a first color among theincident light; and the second image sensor photoelectrically convertslight of a complementary color to the first color.

According to the 3rd aspect of the present invention, it is preferablethat in the imaging device according to the 1st or 2nd aspect, the firstimage sensor comprises a plurality of the first pixels arranged twodimensionally; and the second image sensor comprises a plurality of thefirst pixels arranged two dimensionally; and the imaging device furthercomprises: a first readout unit that reads out signals from a pluralityof the first pixels that are arranged in the second direction; and asecond readout unit that reads out signals from a plurality of thesecond pixels that are arranged in the first direction.

According to the 4th aspect of the present invention, the imaging deviceaccording to any one of the 1st through 3rd aspects may furthercomprise: a micro lens that has a focal point between the first imagesensor and the second image sensor, and that is disposed at a lightincident side of each of the first pixels.

According to the 5th aspect of the present invention, the imaging deviceaccording to any one of the 1st through 3rd aspects may furthercomprise: a micro lens that has a focal point upon the first imagesensor or upon the second image sensor, and that is disposed at a lightincident side of each of the first pixels.

According to the 6th aspect of the present invention, the imaging deviceaccording to any one of the 1st through 3rd aspects may furthercomprise: a micro lens disposed at a light incident side of each of thefirst pixels; and an inner lens disposed between the first image sensorand the second image sensor for each of the second pixels.

According to the 7th aspect of the present invention, it is preferablethat in the imaging device according to any one of the 1st through 3rdaspects, the first through fourth photoelectric conversion units areorganic photoelectric sheets.

According to the 8th aspect of the present invention, it is preferablethat in the imaging device according to any one of the 1st through 3rdaspects, the incident light is a light flux that has passed through aphotographic optical system; and the imaging device further comprises: afocus detection unit that detects the focus adjustment state of thephotographic optical system on the basis of a first signal from thefirst photoelectric conversion unit and a second signal from the secondphotoelectric conversion unit, and that detects the focus adjustmentstate of the photographic optical system on the basis of a third signalfrom the third photoelectric conversion unit and a fourth signal fromthe fourth photoelectric conversion unit; and an image signal generationunit that generates an image signal by adding together the first signaland the second signal, and generates an image signal by adding togetherthe third signal and the fourth signal.

According to the 9th aspect of the present invention, the imaging deviceaccording to the 8th aspect may further comprise: a selection unit thatselects whether to perform focus adjustment of the photographic opticalsystem on the basis of the focus adjustment state detected by the focusdetection unit on the basis of the first and second signals, or toperform focus adjustment of the photographic optical system on the basisof the focus adjustment state detected by the focus detection unit onthe basis of the third and fourth signals.

According to the 10th aspect of the present invention, the imagingdevice according to the 8th aspect may further comprise: a combiningunit that combines together an image signal generated by the imagesignal generation unit by adding together the first signal and thesecond signal, and an image signal generated by the image signalgeneration unit by adding together the third signal and the fourthsignal.

According to the 11th aspect of the present invention, the imagingdevice according to any one of the 1st through 3rd aspects may furthercomprise: a moving direction detection unit that detects a direction ofmoving of a photographic subject according to at least one of signalsfrom the first pixels and signals from the second pixels; and an imagesignal selection unit that selects one of an image signal due to thesignals from the first pixels and an image signal due to the signalsfrom the second pixels, on the basis of the direction of moving of thephotographic subject detected by the moving direction detection unit.

According to the 12th aspect of the present invention, the imagingdevice of any one of the 1st through 3rd aspects may further comprise: amoving direction detection unit that detects a direction of moving of aphotographic subject according to at least one of signals from the firstpixels and signals from the second pixels; and a correction unit thatcorrects one of a first image due to the signals from the first pixelsand a second image due to the signals from the second pixels, on thebasis of the direction of moving of the photographic subject detected bythe moving direction detection unit.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a figure showing an example of the structure of a digitalcamera according to a first embodiment;

FIG. 2 is a figure showing general sketches of a first and a secondimage sensor;

FIG. 3A is a figure showing the configuration of a portion of the firstimage sensor that is 10 rows by 6 columns, and FIG. 3B is a figureshowing the configuration of a portion of the second image sensor thatis 10 rows by 6 columns;

FIG. 4 is a sectional view showing the structure of pixels of the firstand second image sensors;

FIG. 5 is a figure showing a simplified readout circuit for thephotoelectrically converted signals of the first and secondphotoelectric conversion units of the pixels of the first image sensor;

FIG. 6 is a figure showing a simplified readout circuit for thephotoelectrically converted signals of the first and secondphotoelectric conversion units of the pixels of the second image sensor;

FIG. 7 is a block diagram showing in detail the functions fulfilled by afocus detection unit shown in FIG. 1;

FIG. 8 is a flow chart showing first focus detection operation;

FIG. 9 is a block diagram showing a variant of the first embodiment;

FIG. 10 is a sectional view showing a variant embodiment of thestructures of the first and second image sensors;

FIG. 11 is a sectional view showing another variant embodiment of thestructures of the first and second image sensors;

FIG. 12 is a sectional view showing yet another variant embodiment ofthe structures of the first and second image sensors;

FIG. 13 is a sectional view showing still another variant embodiment ofthe structures of the first and second image sensors;

FIG. 14 is a block diagram showing a variant of the first embodiment;

FIG. 15A is a figure showing a variant embodiment of the pixels of thefirst image sensor, and FIG. 15B is a figure showing a variantembodiment of the pixels of the second image sensor;

FIG. 16A is a figure showing a variant embodiment of the pixels of thefirst and second image sensors, and FIG. 16B is a figure showing anothervariant embodiment of the pixels of the first and second image sensors;

FIGS. 17A-17B are figures for explanation of the fundamental concept ofa second embodiment: FIG. 17A is a figure showing a pixel of a firstimage sensor; FIG. 17B is a figure showing a pixel of a second imagesensor; and FIG. 17C is a figure in which pixels of the first and secondimage sensors that are in a relationship of correspondence are shown asmutually superimposed;

FIGS. 18A-18D are figures showing a method for combination of first andsecond photoelectrically converted signals of pixels that are in arelationship of mutual correspondence relating to the first and secondimage sensors according to this second embodiment: FIG. 18A is a figureshowing pixels of a portion of the first image sensor, FIG. 18B is afigure showing the relationship between RGB image signals and the pixelsof the first image sensor, FIG. 18C is a figure showing pixels of aportion of the second image sensor, and FIG. 18D is a figure in whichthese pixels of the first and second image sensors that are in acorresponding relationship are shown as mutually superimposed, andschematically shows the relationship between the overlapped pixels andtheir added signals;

FIGS. 19A-19E are figures for explanation of the theory of a thirdembodiment: FIG. 19A is an image that has been obtained by capturing animage of a photographic subject that is stationary, FIG. 19B is an imagethat has been obtained by capturing, with a first image sensor, an imageof a photographic subject that is moving in the direction of an arrowsign, FIG. 19C is an image that has been obtained by capturing, with thefirst image sensor, an image of a photographic subject that is moving inthe direction of an arrow sign, FIG. 19D is an image that has beenobtained by capturing, with a second image sensor, an image of aphotographic subject that is moving in the direction of an arrow sign,and FIG. 19E is an image that has been obtained by capturing, with thesecond image sensor, an image of a photographic subject that is movingin the direction of an arrow sign; and

FIG. 20 is a block diagram of this third embodiment.

DESCRIPTION OF EMBODIMENTS Embodiment #1

FIG. 1 is a figure showing an example of the structure of a digitalcamera 1 according to the first embodiment. This digital camera 1comprises a photographic optical system 10, an image capturing unit 11,a control unit 12, an actuation unit 13, an image processing unit 14, adisplay such as a liquid crystal or an EL or the like, in other words amonitor 15, and a buffer memory 16. Moreover, a memory card 17 isinstalled to the digital camera 1. This memory card 17 incorporates anon-volatile flash memory or the like, and can be removable from thedigital camera 1.

The photographic optical system 10 comprises a plurality of lenses, andformes an image of a photographic subject upon a photographic imagesurface of an image capturing unit 11. A focusing lens that is drivenalong the direction of the optical axis for focus adjustment is includedin the plurality of lenses that make up the photographic optical system10. This focusing lens is driven along the direction of the optical axisby a lens drive unit not shown in the figures.

The image capturing unit 11 comprises a first image sensor 21 and asecond image sensor 22 that are mutually laminated together, anamplification circuit 23, and an A/D conversion circuit 24. Each of thefirst and second image sensors 21, 22 is built from a plurality ofpixels that are arranged two dimensionally, and receives incident light,in other words a light flux of visible light, from the photographicsubject via the photographic optical system 10, performs photoelectricconversion upon that light, and outputs a photoelectrically convertedsignal. As will be described hereinafter, each of the pixels of thefirst and second image sensors 21, 22 comprises first and secondphotoelectric conversion units that respectively receive a pair of lightfluxes that have passed through a pair of regions of the pupil of thephotographic optical system 10, and these first and second photoelectricconversion units of each pixel respectively output first and secondphotoelectrically converted analog signals. And, as will be describedhereinafter, along with the first and second photoelectrically convertedsignals being employed as signals for phase difference method focusdetection, they are also employed as signals for imaging.

The amplification circuit 23 amplifies the first and secondphotoelectrically converted signals by predetermined amplificationratios (i.e., gains) and outputs the results to the A/D conversioncircuit 24. And the A/D conversion circuit 24 performs A/D conversionupon the first and second photoelectrically converted signals.

The control unit 12 comprises a microprocessor and peripheral circuitrythereof, and performs various types of control for the digital camera 1by executing a control program that is stored in a ROM not shown in thefigures. Moreover, the control unit 12 is endowed with the functions ofa focus detection unit 12 a, an image generation unit 12 b, and a focusdetection area setting unit 12 c. Each of these functional units 12 a,12 b, and 12 c is implemented in software by the control programdescribed above. It should be understood that it would also beacceptable for these functional units to be implemented by electroniccircuitry.

The control unit 12 stores the first and second photoelectricallyconverted signals that have been A/D converted by the A/D conversioncircuit 24 in the buffer memory 16. And the focus detection unit 12 adetects the focus adjustment state of the photographic optical system 10on the basis of the first and second photoelectrically converted signalsof the first image sensor 21 stored in the buffer memory 16, and on thebasis of the first and second photoelectrically converted signals of thesecond image sensor 22 stored in the buffer memory 16.

The image generation unit 12 b generates image signals on the basis ofthe first and second photoelectrically converted signals of both thefirst and second image sensors 21, 22 stored in the buffer memory 16. Inother words, the image generation unit 12 b generates first imagesignals by adding together the first and second photoelectricallyconverted signals of the pixels of the first image sensor 21, and alsogenerates second image signals by adding together the first and secondphotoelectrically converted signals of the pixels of the second imagesensor 22.

The image processing unit 14 incorporates an ASIC and so on. The imageprocessing unit 14 performs image processing of various kinds upon thefirst and second image signals from the image generation unit 12 b, suchas interpolation processing, compression processing, white balanceprocessing and so on, and generates first and second image data. Thisfirst and second image data may be displayed upon the monitor 15, and/ormay be stored upon the memory card 17.

The actuation unit 13 comprises actuation members of various types, suchas a release actuation member, a mode changeover actuation member, anactuation member that is used for setting a focus detection area, apower supply actuation member, and so on, and is actuated by thephotographer. And the actuation unit 13 outputs actuation signals to thecontrol unit 12 according to actuation by the photographer of thevarious actuation members described above.

And, according to actuation of the actuation member for focus detectionarea setting, the focus detection area setting unit 12 c sets the focusdetection area of a predetermined region within the photographic scene.As will be described in detail hereinafter, various possible methods areavailable as the setting method for the focus detection area by thefocus detection area setting unit 12 c and the actuation member forfocus detection area setting. For example, when the photographeractuates the actuation member for focus detection area setting, thefocus detection area setting unit 12 c may set the focus detection areawith a predetermined region on any desired position in the photographicscene, according to this actuation of the actuation member for focusdetection area setting. Or alternatively, focus detection areas may beprepared in advance for a plurality of spots in the photographic scene,and, when the photographer actuates the actuation member for focusdetection area setting and selects one focus detection area from thisplurality of focus detection areas, the focus detection area settingunit 12 c may set the focus detection area according to this selection.Furthermore, if the digital camera 1 includes a photographic subjectrecognition unit that recognizes from a captured image, the face of aperson or the like that is the photographic subject, then the focusdetection area setting unit 12 c may set the focus detection area tothis face portion when the face of a person who is the photographicsubject is recognized by the photographic subject recognition unit. Inthis case, the actuation member for focus detection area setting becomesan actuation member for selecting the mode for automatically setting thefocus detection area, according to the result of facial recognition ofthe person who is the photographic subject by the photographic subjectrecognition unit. It should be understood that the focus detection areathat, as described above, has been set by the focus detection areasetting unit 12 c as the focus detection subject region will be termedthe “set focus detection area”.

Explanation of the First and Second Image Sensors 21, 22

FIG. 2 is a figure showing general sketches of the first and secondimage sensors 21, 22 according to this embodiment. The first imagesensor 21 is an image sensor that uses organic photoelectric sheets asphotoelectric conversion units, and the second image sensor 22 is animage sensor that uses photodiodes formed upon a semiconductor substrateas photoelectric conversion units. The first image sensor 21 islaminated over the second image sensor 22, and the first and secondimage sensors 21, 22 are disposed upon the optical path of thephotographic optical system 10 so that the optical axis of thephotographic optical system 10 shown in FIG. 1 passes through thecenters of the photographic image surfaces of each of the first andsecond image sensors 21, 22. It should be understood that although, inFIG. 2, in order to avoid complication, only four rows by three columnsof the pixels 210, 220 of the first and second image sensors 21, 22 areshown, in this first embodiment, both of these elements comprise m rowsby n columns of pixels, and the pixels of the first image sensor 21 areof the same size as the pixels of the second image sensor 22.

Each of the pixels 210 of the first image sensor 210 includes an organicphotoelectric sheet that absorbs (and photoelectrically converts) lightof a predetermined color component. And the color components that arenot absorbed (photoelectrically converted) by the first image sensor 21pass through the first image sensor 21 and are incident upon the secondimage sensor 22, and are photoelectrically converted by the second imagesensor 22. It should be understood that the color components that arephotoelectrically converted by the first image sensor 21 and the colorcomponents that are photoelectrically converted by the second imagesensor 22 are in complementary color relationship. In other words, aswill be described hereinafter, light of the green color component, whichis in a complementary color relationship with magenta, is incident uponthe pixels 220 of the second image sensor 22 that are positioneddirectly behind those pixels, among the pixels 210 of the first imagesensor 21, that absorb and photoelectrically convert the magenta colorcomponent. Similarly, light of the blue color component, which is in acomplementary color relationship with yellow, is incident upon thepixels 220 of the second image sensor 22 that are positioned directlybehind those pixels, among the pixels 210 of the first image sensor 21,that absorb and photoelectrically convert the yellow color component.And light of the red color component, which is in a complementary colorrelationship with cyan, is incident upon the pixels 220 of the secondimage sensor 22 that are positioned directly behind those pixels, amongthe pixels 210 of the first image sensor 21, that absorb andphotoelectrically convert the cyan color component.

In this manner, there is a correspondence relationship between each ofthe pixels 210 of the first image sensor 21 and the pixel 220 of thesecond image sensor that is positioned directly behind that pixel 210,in other words, there is a correspondence relationship between each ofthe pixels 210 of the first image sensor 21 and the pixel 220 of thesecond image sensor 22 that receives the light flux that has passedthrough that pixel 210; and the pixels 210, 220 of the first and secondimage sensors 21, 22 that are in this sort of correspondencerelationship absorb and photoelectrically convert color components thatare in a mutually complementary color relationship. Pixels 210, 220 ofthe first and second image sensors 21, 22 that are in this sort ofcorrespondence relationship will hereinafter be termed “correspondinglyrelated pixels”.

FIGS. 3A and 3B consist of two figures respectively showing theconfiguration of 10 rows by 6 columns of pixels 210 of a portion of thefirst image sensor 21, and the configuration of 10 rows by 6 columns ofpixels 220 of a portion of the second image sensor 22. In FIG. 3A, forthe first image sensor 21, the notation “Mg” that is appended to some ofthe pixels 210 means that those pixels are pixels that absorb andphotoelectrically convert the magenta color component, in other words,means that those pixels are pixels that have spectral sensitivity tomagenta; and, in a similar manner, the notation “Ye” that is appended tosome of the pixels 210 means that those pixels are pixels that absorband photoelectrically convert the yellow color component, in otherwords, means that those pixels are pixels that have spectral sensitivityto yellow; and the notation “Cy” that is appended to some of the pixels210 means that those pixels are pixels that absorb and photoelectricallyconvert the cyan color component, in other words, means that thosepixels are pixels that have spectral sensitivity to cyan. In the oddnumbered pixel rows of this first image sensor 21, “Mg” pixels 210 and“Ye” pixels 210 are arranged in sequence alternatingly; and, in the evennumbered pixel rows, “Cy” pixels 210 and “Mg” pixels 210 are arranged insequence alternatingly.

It should be understood that generally the “Mg” pixels 210 are not ableto absorb 100% of the magenta color component, generally the “Ye” pixels210 are not able to absorb 100% of the yellow color component, andgenerally the “Cy” pixels 210 are not able to absorb 100% of the cyancolor component, and some amounts of those color components inevitablypass through those pixels, although this is not particularly desirable.

In FIG. 3B for the second image sensor 22, the notation “G” that isappended to some of the pixels 220 means that those pixels are pixelsthat absorb and photoelectrically convert the green color component, inother words, means that those pixels are pixels that have spectralsensitivity to green; and, in a similar manner, the notation “B” that isappended to some of the pixels 220 means that those pixels are pixelsthat absorb and photoelectrically convert the blue color component, inother words, means that those pixels are pixels that have spectralsensitivity to blue; and the notation “R” that is appended to some ofthe pixels 220 means that those pixels are pixels that absorb andphotoelectrically convert the red color component, in other words, meansthat those pixels are pixels that have spectral sensitivity to red. Inthe odd numbered pixel rows of this second image sensor 22, “G” pixels220 and “B” pixels 220 are arranged in sequence alternatingly; and inthe even numbered pixel rows, “R” pixels 220 and “G” pixels 220 arearranged in sequence alternatingly. In other words, the pixels in thesecond image sensor 22 are arranged to form a Bayer array.

In FIGS. 3A and 3B, the “Mg” pixels 210 of the first image sensor 21 andthe “G” pixels 220 of the second image sensor 22 are in a mutuallycorresponding relationship, the

“Ye” pixels 210 of the first image sensor 21 and the “B” pixels 220 ofthe second image sensor 22 are in a mutually corresponding relationship,and the “Cy” pixels 210 of the first image sensor 21 and the “R” pixels220 of the second image sensor 22 are in a mutually correspondingrelationship.

In this manner, the first image sensor 21 that is built from organicphotoelectric sheets fulfils the role of a color filter with respect tothe second image sensor 22, and a color image that is complementary tothat of the first image sensor 21 (in the example of FIGS. 3A and 3B, aBayer array image) is received from the second image sensor 22.Accordingly, it is possible to acquire a CMY image that consists of thethree colors Cy, Mg, and Ye from the first image sensor 21, and it ispossible to acquire a RGB image that consists of the three colors R, andB from the second image sensor 22. Since the first image sensor 21operates in this manner as a replacement for a color filter such as isrequired for a prior art image sensor, accordingly it is possible toemploy the incident light which would undesirably be absorbed by a colorfilter more effectively with the first image sensor 21. It should beunderstood that the CMY image from the first image sensor 21 isconverted into an RGB image by per se known color system conversionprocessing as will be described in detail hereinafter, and thus becomesa first image signal.

Next, the positional relationship of the first and second photoelectricconversion units of the pixels 210 of the first image sensor 21 and thepositional relationship of the first and second photoelectric conversionunits of the pixels 220 of the second image sensor 22 will be explained.In FIG. 3A, each pixel 210 of the first image sensor 21 has a firstphotoelectric conversion unit 210 a and a second photoelectricconversion unit 210 b. These first and second photoelectric conversionunits 210 a and 210 b are arranged in the column direction, in otherwords in the vertical direction in FIG. 3A. Moreover, in FIG. 3B, eachpixel 220 of the second image sensor 22 has a first photoelectricconversion unit 220 a and a second photoelectric conversion unit 220 b.These first and second photoelectric conversion units 220 a and 210 bare arranged in the row direction, in other words in the horizontaldirection in FIG. 3B. Thus, as described above, the first and secondphotoelectric conversion units 210 a, 210 b of the pixels 210 of thefirst image sensor 21 and the first and second photoelectric conversionunits 220 a, 220 b of the pixels 220 of the second image sensor 22 arearranged in mutually orthogonal directions.

Moreover, as will be described hereinafter, in the first image sensor21, the first and second photoelectrically converted signals from thefirst and second photoelectric conversion units 210, 210 b of the pixels210 are read out in units of columns. In other words, for example, forthis first image sensor 21, the first and second photoelectricallyconverted signals of the ten pixels 210 positioned in the extreme leftcolumn are read out simultaneously; next, the first and secondphotoelectrically converted signals of the ten pixels 210 positionednext to the extreme left column and on its right are read outsimultaneously; and subsequently, in a similar manner, the first andsecond photoelectrically converted signals of the ten pixels 210positioned in the next column rightward are read out, and so on insequence.

On the other hand, in the second image sensor 22, the first and secondphotoelectrically converted signals from the first and secondphotoelectric conversion units 220, 220 b of the pixels 220 are read outin units of rows. In other words, for example, for this second imagesensor 22, the first and second photoelectrically converted signals ofthe six pixels 220 positioned in the extreme upper row are read outsimultaneously; next, the first and second photoelectrically convertedsignals of the six pixels 220 positioned in the row below that extremeupper row are read out simultaneously; and subsequently, in a similarmanner, the first and second photoelectrically converted signals of thesix pixels 220 positioned in the next lower row are read out, and so onin sequence.

It should be understood that an example of the circuitry for the pixels210 of the first image sensor 21 is, for example, disclosed inInternational Laying-Open Publication 2013/105,481.

FIG. 4 is a sectional view showing the structure of respective pixels210, 220 of the first and second image sensors 21, 22. As shown in FIG.4, the second image sensor 22 is formed upon a semiconductor substrate50, and each of its pixels 220 comprises the first and secondphotoelectric conversion units 220 a, 220 b that are arranged along thevertical direction. And, via a flattening layer 55, the first imagesensor 21 is laminated to the front surface of the second image sensor22, in other words to its upper surface. A wiring layer 51 is providedwithin this flattening layer 55. It should be understood that while, inFIG. 4, the wiring layer 51 has a three-layered structure, it would alsobe acceptable for it to have a two-layered structure.

Each of the pixels 210 of the first image sensor 21 comprises an organicphotoelectric sheet 230, a transparent common electrode 231 that isformed on the upper surface of the organic photoelectric sheet 230, andtransparent first and second partial electrodes 232 a, 232 b that areformed on the lower surface of the organic photoelectric sheet 230. Asdescribed above, the first and second partial electrodes 232 a, 232 bare arranged along the left to right direction on the drawing paper, inother words in the direction orthogonal to the direction along which thefirst and second photoelectric conversion units 220 a, 220 b of thesecond image sensor 22 are arranged. In each of the pixels 210 of thefirst image sensor 21, the organic photoelectric sheet 230, the commonelectrode 231, and the first partial electrode 232 a constitute thefirst photoelectric conversion unit 210 a, while the organicphotoelectric sheet 230, the common electrode 231, and the secondpartial electrode 232 b constitute the second photoelectric conversionunit 210 b.

Moreover, a micro lens 233 is disposed over each of the pixels 210 ofthe first image sensor 21, and each of the micro lenses 233, thecorresponding pixel 210 of the first image sensor 21, and thecorresponding pixel 220 of the second image sensor 22 are arranged inseries along the direction of the optical axis of that micro lens 233.

Furthermore, the focal point 233F of the micro lens 233 is positioned atthe middle between the first image sensor 21 and the second image sensor22. In other words, the distance between the focal plane of the microlens 233 (in other words the plane orthogonal to the optical axis of themicro lens 233 that includes the focal point F) and the first and secondphotoelectric conversion units of the first image sensor 21 is set so asto be equal to the distance between the focal plane of the micro lens233 and the first and second photoelectric conversion units of thesecond image sensor 22. Since the gap between the first and second imagesensors 21, 22 is comparatively small, and since moreover, as describedabove, the focal point 233F of the micro lens 233 is positioned at themiddle between the first image sensor 21 and the second image sensor 22,accordingly the plane that is conjugate to the first and secondphotoelectric conversion units of the first image sensor 21 with respectto the micro lens 233 (hereinafter this conjugate plane will be referredto as the “first focus detection pupil plane”) is located, along thedirection of the optical axis of the photographic optical system 10shown in FIG. 1, in the vicinity of the plane that is conjugate to thefirst and second photoelectric conversion units of the second imagesensor 22 with respect to the micro lens 233 (hereinafter this conjugateplane will be referred to as the “second focus detection pupil plane”).In other words, the first and the second focus detection pupil planesare positioned near one another in the vicinity along the direction ofthe optical axis of the photographic optical system 10.

Since the micro lens 233 and the first and second image sensors 21, 22are arranged as described above, accordingly the first and secondphotoelectric conversion units 210 a, 210 b of each pixel 210 of thefirst image sensor 21 respectively receive a pair of light fluxes thathave respectively passed through first and second pupil regions of thefirst focus detection pupil plane, while the first and secondphotoelectric conversion units 220 a, 220 b of each pixel 220 of thesecond image sensor 22 respectively receive a pair of light fluxes thathave respectively passed through third and fourth pupil regions of thesecond focus detection pupil plane. It should be understood that thedirection along which the first and second pupil regions of the firstfocus detection pupil plane are arranged and the direction along whichthe third and fourth pupil regions of the second focus detection pupilplane are arranged are mutually orthogonal.

On the basis of the first and second photoelectrically converted signalsof the plurality of pixels 210 that are arranged in the column directionof the first image sensor 21, as will be described hereinafter, thefocus detection unit 12 a shown in FIG. 1 detects the amounts ofdeviation between the pairs of images that are formed by the pairs oflight fluxes that have passed through the first and second pupilregions, in other words detects their phase differences, and calculatesa defocus amount on the basis of this amount of image deviation.Moreover, on the basis of the first and second photoelectricallyconverted signals of the plurality of pixels 220 that are arranged inthe row direction of the second image sensor 22, the focus detectionunit 12 a detects the amounts of deviation between the pairs of imagesthat are formed by the pairs of light fluxes that have passed throughthe third and fourth pupil regions, and calculates a defocus amount onthe basis of this amount of image deviation.

Circuit Structure of the Image Sensor 21

FIG. 5 is a figure showing a simplified readout circuit for thephotoelectrically converted signals of the first and secondphotoelectric conversion units 210 a and 210 b of the pixels 210 of thefirst image sensor 21. In FIG. 5, the first image sensor 21 comprises acolumn scan circuit 151 and first and second horizontal output circuits152, 153. The column scan circuit 151 outputs timing signals R(n) forreading out signals from the first and second photoelectric conversionunits 210 a, 210 b of the plurality of pixels 210 that are arranged inthe column direction, in other words in the vertical direction in FIG.5. To describe this in detail, the column scan circuit 151 outputs atiming signal R(1) to the first and second photoelectric conversionunits 210 a, 210 b of the plurality of pixels 210 of the first column.

According to this timing signal R(1), first and second photoelectricallyconverted signals of the first and second photoelectric conversion units210 a, 210 b of the plurality of pixels 210 in the first column aresimultaneously read out by the first and second horizontal outputcircuits 152, 153 respectively. In this first embodiment, the firstphotoelectrically converted signals of the first photoelectricconversion units 210 a are read out by the first horizontal outputcircuit 152, and the second photoelectrically converted signals of thesecond photoelectric conversion units 210 b are read out by the secondhorizontal output circuit 153. The first horizontal output circuit 152outputs the first photoelectrically converted signals of the firstphotoelectric conversion units 210 a of the first column of pixels,which have thus been read out, sequentially from an output unit 152A,and, in a similar manner, the second horizontal output circuit 153outputs the second photoelectrically converted signals of the secondphotoelectric conversion units 210 b of the first column of pixels,which have thus been read out, sequentially from an output unit 153A.

Next, the column scan circuit 151 outputs a timing signal R(2) to thefirst and second photoelectric conversion units 210 a, 210 b of theplurality of pixels 210 of the second column. According to this timingsignal R(2), the first and second photoelectrically converted signals ofthe first and second photoelectric conversion units 210 a, 210 b of theplurality of pixels 210 in the second column are simultaneously read outby the first and second horizontal output circuits 152, 153respectively. And the first and second horizontal output circuits 152,153 respectively output the first and second photoelectrically convertedsignals of the first and second photoelectric conversion units 210 a,210 b of the second column of pixels, which have thus been read out,sequentially from the output units 152A, 153A.

In the following, in a similar manner, the column scan circuit 151outputs timing signals R(n) to the first and second photoelectricconversion units 210 a, 210 b of the plurality of pixels 210 of the n-thcolumn successively. And, according to these timing signals R(n), thefirst and second photoelectrically converted signals of the first andsecond photoelectric conversion units 210 a, 210 b of the plurality ofpixels 210 in the n-th column are simultaneously read out by the firstand second horizontal output circuits 152, 153 respectively, and aresequentially outputted from the output units 152A, 153A of the first andsecond horizontal output circuits 152, 153.

The first photoelectrically converted signals outputted from the firsthorizontal output circuit 152 and the second photoelectrically convertedsignals outputted from the second horizontal output circuit 153 are sentvia the buffer memory 16 to the focus detection unit 12 a and to theimage generation unit 12 b shown in FIG. 1, and the focus detection unit12 a performs phase difference focus detection calculation on the basisof the first and second photoelectrically converted signals of the firstand second photoelectric conversion units 210 a, 210 b of the n-thcolumn that have thus been simultaneously read out. Moreover, the imagegeneration unit 12 b adds together the photoelectrically convertedsignals of the first and second photoelectric conversion units 210 a,210 b of each pixel 210, and thereby generates an image signal.

FIG. 6 is a figure showing a simplified readout circuit for thephotoelectrically converted signals of the first and secondphotoelectric conversion units 220 a and 220 b of the pixels of thesecond image sensor 22. In FIG. 6, the second image sensor 22 comprisesa row scan circuit 161 and first and second horizontal output circuits162, 163. Since the signal readout circuits 161, 162, and 163 related tothe photoelectrically converted signals of the first and secondphotoelectric conversion units 220 a, 220 b of the second image sensor22 are similar to the signal readout circuits 151, 152, and 153 of thefirst image sensor 22 shown in FIG. 5, only the aspects in which theydiffer will be explained below.

The row scan circuit 161 outputs timing signals R(m) for reading outsignals from the first and second photoelectric conversion units 220 a,220 b of the pluralities of pixels 220 that are arranged in the rowdirection, i.e. in the horizontal direction in FIG. 6. In other words,the row scan circuit 161 outputs a timing signal R(1) for reading outsignals from the first and second photoelectric conversion units 220 a,220 b of the plurality of pixels 220 in the first row, then next outputsa timing signal R(2) for reading out signals from the first and secondphotoelectric conversion units 220 a, 220 b of the plurality of pixels220 in the second row, and thereafter sequentially outputs timingsignals R(m) for reading out signals from the first and secondphotoelectric conversion units 220 a, 220 b of the plurality of pixels220 in the m-th row.

According to this timing signal R(m), the first horizontal outputcircuit 162 simultaneously reads out the first photoelectricallyconverted signals of the first photoelectric conversion units 220 a ofthe plurality of pixels 220 in the m-th row, and, in a similar manner,the second horizontal output circuit 163 simultaneously reads out thesecond photoelectrically converted signals of the second photoelectricconversion units 220 b of the plurality of pixels 220 in the m-th row.

The first horizontal output circuit 162 outputs the firstphotoelectrically converted signals of the first photoelectricconversion unit 220 a, which have thus been read out, to the output unit162A, and the second horizontal output circuit 163 outputs the secondpotoelectrically converted signals of the second photoelectricconversion unit 220 b, which have thus been read out, to the output unit163A.

The first photoelectrically converted signals outputted from the firsthorizontal output circuit 162 and the second photoelectrically convertedsignals outputted from the second horizontal output circuit 163 are sentvia the buffer memory 16 to the focus detection unit 12 a and to theimage generation unit 12 b shown in FIG. 1, and the focus detection unit12 a performs phase difference focus detection calculation on the basisof the first and second photoelectrically converted signals of the firstand second photoelectric conversion units 220 a, 220 b of the m-th rowthat have been simultaneously read out. Moreover, the image generationunit 12 b adds together the first and second photoelectrically convertedsignals of the first and second photoelectric conversion units 210 a,220 b of each pixel 220, and thereby generates an image signal.

It should be understood that the signal readout circuit example of thefirst image sensor 21 shown in FIG. 5 and the signal readout circuitexample of the second image sensor 22 shown in FIG. 6 are both formedupon the semiconductor substrate 50 of the second image sensor 22 shownin FIG. 4, and the pixels 210, 220 of the respective first and secondimage sensors 21, 22 are connected to the wiring layer 51.

FIG. 7 is a block diagram showing in detail the functions fulfilled bythe focus detection unit 12 a shown in FIG. 1. The focus detection unit12 a comprises first and second focus detection signal acquisition units120 and 121, first and second contrast detection units 122 and 123, adecision unit 124, a selection unit 125, a correlation calculation unit126, and a defocus amount calculation unit 127.

The focus detection area setting unit 12 c outputs position informationrelated to the set focus detection area that has been set as the focusdetection subject region. For the first image sensor 21, the first focusdetection signal acquisition unit 120 acquires, from among the first andsecond photoelectrically converted signals outputted from the first andsecond horizontal output circuits 152, 153 shown in FIG. 5, first andsecond photoelectrically converted signals from a plurality of pixels210 that are arranged in the column direction in positions correspondingto the set focus detection area that has been set by the focus detectionarea setting unit 12 c.

In a similar manner, for the second image sensor 22, the second focusdetection signal acquisition unit 121 acquires, from among the first andsecond photoelectrically converted signals outputted from the first andsecond horizontal output circuits 162, 163 shown in FIG. 6, first andsecond photoelectrically converted signals from a plurality of pixels220 that are arranged in the row direction in positions corresponding tothe set focus detection area that has been set by the focus detectionarea setting unit 12 c.

Due to the above, for the first image sensor 21, the first focusdetection signal acquisition unit 120 acquires the first and secondphotoelectrically converted signals that were simultaneously read outfrom the plurality of pixels 210 arranged in the column direction andcorresponding to the set focus detection area. These first and secondphotoelectrically converted signals related to the first image sensor 21that have been acquired by the first focus detection signal acquisitionunit 120 will be termed the “first focus detection signals”. Moreover,for the second image sensor 22, the second focus detection signalacquisition unit 121 acquires the first and second photoelectricallyconverted signals that were simultaneously read out from the pluralityof pixels 220 arranged in the row direction and corresponding to the setfocus detection area. These first and second photoelectrically convertedsignals related to the second image sensor 22 that have been acquired bythe second focus detection signal acquisition unit 121 will be termedthe “second focus detection signals”.

It should be understood that the first and second focus detectionsignals are each created from respective first and secondphotoelectrically converted signals of pixels that have the samespectral sensitivity. In concrete terms, the first focus detectionsignals related to the first image sensor 21 are selected from the firstand second photoelectrically converted signals of the Mg pixels 210 thatare arranged in the column direction in FIG. 3A as each second pixel. Ina similar manner, the second focus detection signals related to thesecond image sensor 22 are selected from the first and secondphotoelectrically converted signals of the G pixels 220 that arearranged in the row direction in FIG. 3B as each second pixel. Ofcourse, the first focus detection signals are not limited to being thefirst and second photoelectrically converted signals of the Mg pixels,and it would also be possible to employ the first and secondphotoelectrically converted signals of the Cy pixels or of the Yepixels; and similarly the second focus detection signals are not limitedto being the first and second photoelectrically converted signals of theG pixels, and it would also be possible to employ the first and secondphotoelectrically converted signals of the R pixels or of the B pixels.

The first contrast detection unit 122 calculates a first contrast amountof the image of the photographic subject in the set focus detection areaon the basis of the first focus detection signals that have beenacquired by the first focus detection signal acquisition unit 120. Itshould be understood that this contrast amount is calculated byintegrating the difference between the first photoelectrically convertedsignals or the second photoelectrically converted signals (or betweenthe totals of the first and the second photoelectrically convertedsignals) related to adjacent pixels that have been acquired by the firstfocus detection signal acquisition unit 120. It should be understoodthat this first contrast amount is the contrast amount of the image ofthe photographic subject that has been created upon the plurality ofpixels 210 arranged in the column direction within the set focusdetection area of the first image sensor 21.

And, in a similar manner to the case of the first contrast detectionunit 122, the second contrast detection unit 123 calculates a secondcontrast amount of the image of the photographic subject in the setfocus detection area on the basis of the second focus detection signalsthat have been acquired by the second focus detection signal acquisitionunit 121. This second contrast amount is the contrast amount of theimage of the photographic subject that has been created upon theplurality of pixels 220 arranged in the row direction within the setarea of the second image sensor 22.

The decision unit 124 makes a decision as to whether or not at least oneof the first and second contrast amounts is greater than or equal to afirst threshold value. This first threshold value is determined so thatthe focus detection signals whose the contrast amount that is greaterthan or equal to the first threshold value can be employed in aneffective manner. Accordingly, if both the first contrast amount and thesecond contrast amount are below the first threshold value, then thedecision unit 124 decides that the image of the photographic subject inthe set focus detection area is extremely blurred, in other words thatit is in an extremely highly defocused state, and accordingly performsscanning driving of the focusing lens of the photographic lens 10.

But if at least one of the first and second contrast amounts is greaterthan or equal to the first threshold value, then the decision unit 124makes a decision as to whether or not the difference between the firstcontrast amount and the second contrast amount is greater than or equalto a second threshold value. If the difference between the firstcontrast amount and the second contrast amount is greater than or equalto a second threshold value, then the decision unit 124 determines that,among the first and second contrast amounts, the focus detection signalswhose contrast amount is the larger will be applied for phase differencefocus detection, but that the focus detection signals whose contrastamount is the smaller will not be applied for phase difference focusdetection. However, if the difference between the first contrast amountand the second contrast amount is smaller than the second thresholdvalue, then the decision unit 124 determines that both the first and thesecond focus detection signals will be applied for phase differencefocus detection.

On the basis of the output signal from the decision unit 124, theselection unit 125 selects either one or both of the first and secondfocus detection signals of the first and second focus detection signalacquisition units 120, 121, and sends the result to the correlationcalculation unit 126. To explain this in detail, if the differencebetween the first contrast amount and the second contrast amount isgreater than or equal to the second threshold value, then, when thefirst contrast amount is greater than the second contrast amount, theselection unit 125 selects the first focus detection signals of thefirst focus detection signal acquisition unit 120 and outputs them tothe correlation calculation unit 126; and, when the second contrastamount is greater than the first contrast amount, the selection unit 125selects the second focus detection signals of the second focus detectionsignal acquisition unit 121 and outputs them to the correlationcalculation unit 126. Moreover, if the difference between the firstcontrast amount and the second contrast amount is less than the secondthreshold value, then the selection unit 125 selects both the firstfocus detection signals and the second focus detection signals of thefirst and second focus detection signal acquisition units 120, 121 andoutputs them to the correlation calculation unit 126.

If the first focus detection signals are inputted from the selectionunit 125, then the correlation calculation unit 126 performs correlationcalculation on the basis of these first focus detection signals andcalculates a first amount of image deviation, while, if the second focusdetection signals are inputted from the selection unit 125, then thecorrelation calculation unit 126 performs correlation calculation on thebasis of these second focus detection signals and calculates a secondamount of image deviation; and, if both the first and the second focusdetection signals are inputted from the selection unit 125, then thecorrelation calculation unit 126, along with performing correlationcalculation on the basis of the first focus detection signals andcalculating a first amount of image deviation, also performs correlationcalculation on the basis of the second focus detection signals andcalculates a second amount of image deviation.

The defocus amount calculation unit 127 calculates a defocus amount onthe basis of the results of correlation calculation by the correlationcalculation unit 126, in other words on the basis of the amount of imagedeviation. And focus adjustment of the photographic optical system isperformed on the basis of this defocus amount. It should be understoodthat if, as described above, the selection unit 125 has selected boththe first and second focus detection signals, then the defocus amountcalculation unit 127 calculates the average value of a first defocusamount based upon the first focus detection signals and a second defocusamount based upon the second focus detection signals, and takes thisaverage value as a final defocus amount. And focus adjustment of thephotographic optical system is performed on the basis of this finaldefocus amount.

In this manner, the readout circuit is built so that the first andsecond photoelectrically converted signals are both read outsimultaneously from a plurality of pixels 210 that are arranged in thecolumn direction of the first image sensor 21, and so that the phasedifference focus detection calculation is performed on the basis ofthese first and second photoelectrically converted signals that haveboth been read out simultaneously, in other words on the basis of thefirst focus detection signals. In a similar manner, the readout circuitis built so that the first and second photoelectrically convertedsignals are both read out simultaneously from a plurality of pixels 220that are arranged in the row direction of the second image sensor 22,and so that the phase difference focus detection calculation isperformed on the basis of these first and second photoelectricallyconverted signals that have both been read out simultaneously, in otherwords on the basis of the second focus detection signals. Since, due tothis, the accuracy of calculation of the defocus amount becomes higherwhen the phase difference focus detection calculation is performed withthe first focus detection signals that are read out simultaneously, orwith the second focus detection signals that are read outsimultaneously, accordingly it becomes possible to enhance the focusadjustment accuracy of the photographic optical system 10, and itbecomes possible to enhance the image quality of the image that isobtained by image capture.

FIG. 8 is a flow chart showing this first focus detection operation. Thefocus detection processing shown in FIG. 8 is included in the controlprogram executed by the control unit 12. The control unit 12 starts thefocus detection processing shown in FIG. 8 when predetermined focusdetection actuation is performed by the photographer, for example uponhalf press actuation of a release actuation member or the like.

Referring to FIGS. 7 and 8, in step S1 each of the first image sensor 21and the second image sensor 22 captures an image of the photographicsubject that has been formed by the photographic optical system 10. Inthe first image sensor 21, the first and second photoelectricallyconverted signals of the first and second photoelectric conversion units210 a, 210 b of a plurality of pixels 210 that are arranged in thecolumn direction are simultaneously read out. And, in the second imagesensor 22, the first and second photoelectrically converted signals ofthe first and second photoelectric conversion units 220 a, 220 b of aplurality of pixels 220 that are arranged in the row direction aresimultaneously read out. The first and second photoelectricallyconverted signals from the first image sensor 21 or from the secondimage sensor 22 are added together by the image generation unit 12 b ofFIG. 1 to produce a first or a second image signal, and this signal isimage processed by the image processing unit 14 of FIG. 1 and isdisplayed upon the monitor 15 of FIG. 1 as a live view image. It shouldbe understood that it will be acceptable for such a live view image tobe displayed on the basis of the first image signal from the first imagesensor 21, or to be displayed on the basis of the second image signalfrom the second image sensor 22.

In step S2, a set focus detection area within the photographic scene isset by the focus detection area setting unit 12 c of FIG. 1. And then instep S3 the first focus detection signal acquisition unit 120 acquires,as the first focus detection signals, first and second photoelectricallyconverted signals of the first and second photoelectric conversion units210 a, 210 b, among the first and second photoelectrically convertedsignals outputted from the first image sensor 21, of the plurality ofpixels 210 that are arranged in the column direction and that correspondto the set focus detection area. And, in a similar manner the secondfocus detection signal acquisition unit 121 acquires, as the secondfocus detection signals, first and second photoelectrically convertedsignals of the first and second photoelectric conversion units 220 a,220 b, among the first and second photoelectrically converted signalsoutputted from the second image sensor 22, for the plurality of pixels220 that are arranged in the row direction and that correspond to theset focus detection area.

In step S4, the first contrast detection unit 122 calculates a firstcontrast amount of the image of the photographic subject in the setfocus detection area on the basis of the first focus detection signalsacquired by the first focus detection signal acquisition unit 120. Asdescribed above, this first contrast amount is the contrast amount inthe column direction of the pixels 210 of the first image sensor 21. Ina similar manner, the second contrast detection unit 123 calculates asecond contrast amount of the image of the photographic subject in theset focus detection area on the basis of the second focus detectionsignals acquired by the second focus detection signal acquisition unit121. As described above, this second contrast amount is the contrastamount in the column direction of the pixels 220 of the second imagesensor 22.

Then in step S5 the decision unit 124 decides whether or not at leastone of the first contrast amount and the second contrast amount isgreater than or equal to the first threshold value, and if a negativedecision is reached then the flow of control proceeds to step S6, whileif an affirmative decision is reached then the flow of control istransferred to step S7. In step S6, it is decided that the image of thephotographic subject in the set focus detection area is extremelyblurred, in other words is in an extremely highly defocused state, andthe focusing lens of the photographic lens 10 is driven to performscanning. On the other hand, in step S7 the decision unit 124 decideswhether or not the difference between the first contrast amount and thesecond contrast amount is greater than or equal to the second thresholdvalue, and if an affirmative decision is reached then the flow ofcontrol proceeds to step S8, while if a negative decision is reachedthen the flow of control is transferred to step S9. In step S8, adecision is made as to whether or not the first contrast amount isgreater than the second contrast amount, and if an affirmative decisionis reached then the flow of control is transferred to step S10, while ifa negative decision is reached then the flow of control is transferredto step S11.

In step S10, since the first contrast amount is greater than the secondcontrast amount, in other words since the contrast amount in the columndirection of the first image sensor 21 is greater than the contrastamount in the row direction of the second image sensor 22 in the setfocus detection area, accordingly the selection unit 124 selects thefirst focus detection signals of the first focus detection signalacquisition unit 120, the correlation calculation unit 126 performscorrelation calculation on the basis of these first focus detectionsignals, and the defocus amount calculation unit 127 calculates adefocus amount on the basis of the result of this correlationcalculation. On the other hand, in step S11, since the second contrastamount is greater than the first contrast amount, in other words sincethe contrast amount in the row direction of the second image sensor 22is greater than the contrast amount of the first image sensor 21 in thecolumn direction in the set focus detection area, accordingly theselection unit 124 selects the second focus detection signals of thesecond focus detection signal acquisition unit 121, the correlationcalculation unit 126 performs correlation calculation on the basis ofthese second focus detection signals, and the defocus amount calculationunit 127 calculates a defocus amount on the basis of the result of thiscorrelation calculation.

Moreover, in step S9, since the difference between the first and secondcontrast amounts is less than the second threshold value, in other wordssince the first and second contrast amounts are almost equal to oneanother, accordingly the selection unit 125 selects both the first andthe second focus detection signals of the first and second focusdetection signal acquisition units 120 and 121, the correlationcalculation unit 126 performs correlation calculation on the basis ofthe first focus detection signals and also performs correlationcalculation on the basis of the second focus detection signals, and thedefocus amount calculation unit 127 calculates a first defocus amount onthe basis of the result of the correlation calculation according to thefirst focus detection signals, and also calculates a second defocusamount on the basis of the result of the correlation calculationaccording to the second focus detection signals, and then calculates afinal defocus amount from the first and second defocus amounts.

In step S12, focus adjustment is performed by driving the focusing lensof the photographic optical system 10 on the basis of the defocus amountcalculated in step S10, S11, or S9. And then in step S13 a decision ismade as to whether or not half press actuation of the release actuationmember has ended, and if an affirmative decision is reached then thefocus detection operation is terminated, whereas if a negative decisionis reached then the flow of control returns to step S1.

Since, as described above, in this first embodiment, the direction ofarrangement of the first and second photoelectric conversion units 210a, 210 b of the pixels 210 of the first image sensor 21 and thedirection of arrangement of the first and second photoelectricconversion units 220 a, 220 b of the pixels 220 of the second imagesensor 22 are different, accordingly the contrast of the image of thephotographic subject that is formed upon the plurality of pixels 210 ofthe first image sensor 21 that are arranged in the column direction inthe set focus detection area and the contrast of the image of thephotographic subject that is formed upon the plurality of pixels 220 ofthe second image sensor 22 that are arranged in the row direction in theset focus detection area are compared together, so that it is possibleto perform focus detection at high accuracy on the basis of the focusdetection signal of that image sensor whose contrast is the higher.

In this first embodiment, according to the first and second contrastamounts, it is arranged to select either one or both of the first andthe second focus detection signals, and for the correlation calculationunit 126 to perform correlation calculation on the basis of the selectedfocus detection signals. Instead of this, it would also be acceptable toarrange for the selection unit 125 always to select the first and thesecond focus detection signals, for the correlation calculation unit 126to perform correlation calculation for both of the first and secondfocus detection signals, to select one or both of the result ofcorrelation calculation according to the first focus detection signalsand the result of correlation calculation according to the second focusdetection signals on the basis of the quantity of the first and secondcontrast amounts, and to calculate a defocus amount on the basis of theresult of the correlation calculation that has thus been selected.

Furthermore, it would also be acceptable to arrange for the defocusamount calculation unit 127 to calculate respective defocus amounts onthe basis of all of the results of correlation calculation calculated bythe correlation calculation unit 126, and, on the basis of the quantityof the first and second contrast amounts, to select an appropriatedefocus amount from the plurality of defocus amounts that have beencalculated in this manner.

With an image sensor in which pixels whose directions of subdivision ofthe first and second photoelectric conversion units are different areprovided upon the photographic image surface, even if the same amountsof light are respectively incident upon two pixels whose directions ofsubdivision of the first and second photoelectric conversion units aredifferent, due to the difference in the directions of subdivision of thefirst and second photoelectric conversion units, there is a danger thatthe outputs from each pixel of the first and second photoelectricconversion units may be different. Due to this, there is a danger ofdeterioration of the quality of the image that is obtained by imagecapture.

By contrast, in this first embodiment, it is arranged to generate imagesignals by adding together the photoelectrically converted signals ofthe first and second photoelectric conversion units 210 a, 210 b of thepixels 210 of the first image sensor 21 that have been subdivided in thecolumn direction. And it is arranged also to generate image signals byadding together the photoelectrically converted signals of the first andsecond photoelectric conversion units 220 a, 220 b of the pixels 220 ofthe second image sensor 22 that have been subdivided in the rowdirection. Due to this, it is possible to obtain an image of high imagequality with the first and second image sensors 21, 22.

Variant Embodiment #1

In this first embodiment, since there is a danger that it will not bepossible to perform phase difference focus detection with good accuracyfor a photographic subject image whose contrast is low, accordingly thecontrast of the image of the photographic subject formed on a pluralityof pixels 210 of the first image sensor 21 that are arranged in thecolumn direction in the set focus detection area, and the contrast ofthe image of the photographic subject formed on a plurality of pixels220 of the second image sensor 22 that are arranged in the row directionin the set focus detection area, are compared together, and focusadjustment is performed on the basis of the focus detection signals ofthat image sensor whose contrast is the higher. Photographic subjectimages for which it is not possible to perform phase difference focusdetection with good accuracy not only are low contrast images asdescribed above, but also are images containing patterns of brightnessand darkness at a fixed cycle. Accordingly, variants of the above firstembodiment will be explained below in which it is detected for which ofthe column direction and the row direction of the pixels of the firstand second image sensors 21, 22 this type of fixed cycle pattern ispresent in the image, and focus detection is performed by employing thefocus detection signals of that one of the image sensors in which such acycle pattern is not present.

FIG. 9 is a block diagram showing a variant of the first embodiment, inwhich the point of difference from the block diagram of the firstembodiment shown in FIG. 7 is that first and second cycle patterndetection units 128, 129 are provided instead of the first and secondcontrast detection units 122, 123 of FIG. 7. The first cycle patterndetection unit 128 detects whether or not a cycle pattern is present inthe first focus detection signals by detecting a cycle pattern signalwaveform in the first focus detection signals from the first focusdetection signal acquisition unit 120. In a similar manner, the secondcycle pattern detection unit 129 detects whether or not a cycle patternis present in the second focus detection signals by detecting a cyclepattern signal waveform in the second focus detection signals from thesecond focus detection signal acquisition unit 121. And the decisionunit 124 makes a decision as to whether or not a cycle pattern has beendetected by either of the first and second cycle pattern detection units128, 129. If the decision unit 124 has decided that neither of the firstand second cycle pattern detection units 128, 129 has detected any cyclepattern, then the selection unit 125 selects both of the first andsecond focus detection signals; while, if the decision unit 124 hasdecided that the first cycle pattern detection unit 128 has detected acycle pattern, then the selection unit 125 selects the second focusdetection signal; and, if the decision unit 124 has decided that thesecond cycle pattern detection unit 129 has detected a cycle pattern,then the selection unit 125 selects the first focus detection signal.The operation of the correlation calculation unit 126 and of the defocusamount calculation unit 127is the same as in the case of FIG. 7.

As has been described above, in this variant embodiment, since thedirection of arrangement of the first and second photoelectricconversion units 210 a, 210 b of the pixels 210 of the first imagesensor 21 and the direction of arrangement of the first and secondphotoelectric conversion units 220 a, 220 b of the pixels 220 of thesecond image sensor 22 are different, accordingly it is possible toperform focus detection at high accuracy on the basis of the focusdetection signal of that image sensor for which no cycle pattern exists,on the basis of the presence or absence of any cycle pattern in theimage of the photographic subject formed upon the plurality of pixels210 that are arranged in the column direction of the first image sensor21 in the set focus direction area, and on the basis of the presence orabsence of any cycle pattern in the image of the photographic subjectformed upon the plurality of pixels 220 that are arranged in the rowdirection of the second image sensor 22 in the set focus direction area.

It should be understood that, in the variant embodiment shown in FIG. 9as well, instead of the selection unit 125 selecting either or both ofthe first and second focus detection signals according to the presenceor absence of cycle patterns, it would also be acceptable for thecorrelation calculation unit 126 always to perform correlationcalculations on the basis of the first and second focus detectionsignals, and to select a correlation calculation result from thosecorrelation calculation results that is determined according to thepresence or absence of cycle patterns; or it would also be acceptable toarrange for the defocus amount calculation unit 127 always to calculatea defocus amount on the basis of the first and second focus detectionsignals, and to arrange to select a defocus amount from those defocusamounts that is determined according to the presence or absence of cyclepatterns.

While, with the first embodiment or the first variant embodimentdescribed above, the first focus detection signals from the first imagesensor 21 and/or the second focus detection signals from the secondimage sensor 22 were selected upon the basis of the quantity of thecontrast amounts or upon the basis of the presence or absence of cyclepatterns therein, instead of the above, it would also be possible todetect the attitude of the digital camera 1 with an attitude sensor orthe like, and to select the second focus detection signals from thesecond image sensor 22, in other words to perform focus adjustment usingthe second focus detection signals when the digital camera 1 is in itsnormal horizontal position, in other words when the row direction of thepixel arrangement of the second image sensor 22 of FIG. 3 and thehorizontal direction coincide, while on the other hand, to select thefirst focus detection signals from the first image sensor 21, in otherwords to perform focus adjustment using the first focus detectionsignals when the digital camera 1 is in its vertical position, in otherwords when the column direction of the pixel arrangement of the firstimage sensor 21 of FIG. 3 and the horizontal direction coincide.

Variant Embodiment #2

FIG. 10 shows a second variant of the first embodiment. In this secondvariant embodiment, the focal point 233F of the micro lens 233 ispositioned upon the first and second photoelectric conversion units 220a, 220 b of the second image sensor 22 as shown by the solid lines inFIG. 10; or alternatively, the focal point 233F of the micro lens 233may be positioned upon the first and second photoelectric conversionunits 210 a, 210 b of the first image sensor 21 as shown by the brokenlines.

Variant Embodiment #3

FIG. 11 shows a third variant of the first embodiment. In FIG. 11, thefocal point 233F of the micro lens 233, in other words its focal plane,is located at the position of the first and second photoelectricconversion units 210 a, 210 b of the first image sensor 21. And an innerlens 234 is disposed between the first image sensor 21 and the secondimage sensor 22. The refracting power and the position of arrangement ofthis inner lens 234 are determined so that the position of the first andsecond photoelectric conversion units 210 a, 210 b of the first imagesensor 21 and the position of the first and second photoelectricconversion units 220 a, 220 b of the second image sensor 22 becomeoptically conjugate with respect to this inner lens 234.

Due to the provision of this inner lens 234 in this configuration, theposition conjugate to the position of the first and second photoelectricconversion units 220 a, 220 b of the second image sensor 22 with respectto the micro lens 233 and the inner lens 234, and the position conjugateto the position of the first and second photoelectric conversion units210 a, 210 b of the first image sensor 21 with respect to the micro lens233, coincide with one another. To put this in another manner, due tothis provision and arrangement of the inner lens 234, the position ofthe first focus detection pupil plane relating to the first image sensor21 and the position of the second focus detection pupil plane withrespect to the second image sensor 22 can be made to coincide with oneanother.

Variant Embodiment #4

FIG. 12 is a sectional view showing a fourth variant of the firstembodiment. In the first embodiment, as shown in FIG. 4, the first imagesensor 21 employed organic photoelectric sheets as photoelectricconversion units, and the second image sensor 22 employed photo-diodesas photoelectric conversion units. However, in this fourth variantembodiment, both the first and the second image sensors 21, 22 employorganic photoelectric sheets as photoelectric conversion units.

In FIG. 12, the structure of the first image sensor 21 is the same asthat of the first image sensor 21 shown in FIG. 4. However, the secondimage sensor 23 is formed upon the upper surface of a semiconductorsubstrate 50 with a flattening layer 55 interposed between them, andeach of its pixels 240 comprises first and second photoelectricconversion units 240 a, 240 b that are arranged in the directionperpendicular to the surface of the drawing paper.

Each of the pixels 240 of the second image sensor 23 comprises anorganic photoelectric sheet 250, a transparent common electrode 251 thatis formed upon the lower surface of the organic photoelectric sheet 250,and transparent first and second partial electrodes 252 a, 252 b thatare formed upon the upper surface of the organic photoelectric sheet250. As described above, these first and second partial electrodes 252a, 252 b are arranged in the direction perpendicular to the surface ofthe drawing paper, in other words in the direction that is orthogonal tothe direction in which the first and second partial electrodes 232 a,232 b of the first image sensor 21 are arranged. In each of the pixels240 of the second image sensor 23, a first photoelectric conversion unit240 a includes the organic photoelectric sheet 250, the common electrode251, and the first partial electrode 252 a, and a second photoelectricconversion unit 240 b includes the organic photoelectric sheet 250, thecommon electrode 251, and the second partial electrode 252 b.

An insulating layer 56 is provided between the first image sensor 21 andthe second image sensor 23. And a signal readout circuit for the firstimage sensor 22 and a signal readout circuit for the second image sensor23 are formed upon the semiconductor substrate 50. A wiring layer 51that may have, for example, a three-layered structure is providedbetween the semiconductor substrate 50 and the second image sensor 23.

Since, as described above, according to this fourth variant embodiment,the wiring layer 51 is provided in the gap between the second imagesensor 23 and the semiconductor substrate 50, accordingly acomparatively large gap is required, on the other hand the gap betweenthe first and second image sensors 21, 23 can be made comparativelysmall, since no wiring layer 51 is required there. Accordingly, it ispossible to bring the position of the focus detection pupil planerelated to the first image sensor 21 and the position of the focusdetection pupil plane related to the second image sensor 23 close to oneanother.

Moreover, it would also be acceptable to arrange for the second imagesensor 22 to be an image sensor of the backside-illumination type, asshown in FIG. 13. Since, in a similar manner to the case describedabove, the provision of any wiring layer 51 is not required, accordinglythe gap between the first and second image sensors 21, 22 can be madecomparatively small. Therefore, it is possible to bring the position ofthe focus detection pupil plane related to the first image sensor 21 andthe position of the focus detection pupil plane related to the secondimage sensor 22 close to one another.

Variant Embodiment #5

FIG. 14 is a block diagram showing a fifth variant of the firstembodiment. In the signal readout circuitry for the first and secondimage sensors 21, 22 shown in FIGS. 5 and 6, the first photoelectricallyconverted signal was outputted from the first horizontal output circuits152, 162 and the second photoelectrically converted signal was outputtedfrom the second horizontal output circuits 153, 163, and moreover theA/D conversion circuit 24 performs A/D conversion on the first andsecond photoelectrically converted signals from the first and secondimage sensors 21, 22, and the image generation unit 12 b of the controlunit 12 performs addition of the first and second photoelectricallyconverted signals. However, this fifth variant embodiment includes anA/D conversion unit that performs A/D conversion on the first and secondphotoelectrically converted signals of the pixels 220 of the secondimage sensor 22, a horizontal output circuit for the first and secondphotoelectrically converted signals that outputs both the first andsecond photoelectrically converted signals, an addition circuit thatadds together the first and second photoelectrically converted signals,and a horizontal output circuit for the added signals that outputs theadded signals.

In FIG. 14, the second image sensor 22 comprises a row scan circuit 161,an A/D conversion unit 164, a horizontal output circuit 165 for thefirst and second photoelectrically converted signals, an addition unit166, and a horizontal output circuit 167 for the added signals. In thefollowing explanation, principally the difference from the readoutcircuitry for the photoelectrically converted signals of the secondimage sensor 22 shown in FIG. 6 will be explained.

The A/D conversion unit 164 comprises n ADCs (analog to digitalconversion circuits) 164 a that respectively correspond to the firstphotoelectric conversion units 220 a of n columns of pixels arranged inthe row direction, and n ADCs 164 b that respectively correspond to thesecond photoelectric conversion units 220 b of these pixels 220.

The horizontal output circuit 165 for the first and secondphotoelectrically converted signals comprises n memories 165 a thatrespectively correspond to the n ADCs 164 a of the A/D conversion unit164, and n memories 165 b that respectively correspond to the n ADCs 164b of the A/D conversion unit 164.

The addition unit 166 comprises n digital addition circuits 165 a thatrespectively correspond to the pixels 220 that are arranged in n columnsin the row direction.

The horizontal output circuit 167 for the added signals comprises nmemories 167 a that respectively correspond to the n digital additioncircuits 166 a of the addition unit 166.

According to the timing signal R(1), the photoelectrically convertedsignals of the first and second photoelectric conversion units 220 a,220 b of the n pixels 220 of the first row are simultaneously outputtedto the respectively corresponding ADCs 164 a, 164 b of the A/Dconversion unit 164. The ADCs 164 a, 164 b convert the photoelectricallyconverted signals of the first and second photoelectric conversion units220 a, 220 b that have thus been inputted into respective digitalsignals, which are then respectively outputted to the correspondingmemories 165 a, 165 b of the horizontal output circuit 165 for the firstand second photoelectrically converted signals. And each of the memories165 a, 165 b of the horizontal output circuit 165 for the first andsecond photoelectrically converted signals stores the respective digitalsignal outputted from the ADCs 164 a, 164 b. Then the first and secondphotoelectrically converted signals thus stored in the memories 165 a,165 b of the horizontal output circuit 165 for the first and secondphotoelectrically converted signals are sequentially outputted from theoutput unit 165 a.

Moreover, the digital addition circuits 166 a of the addition unit 166add together the first and second photoelectrically converted signalsthat have respectively been A/D converted by the ADCs 164 a, 164 b ofeach of the pixels 220. And the memories 167 a of the horizontal outputcircuit 167 for the added signals respectively store the added digitalsignals outputted from the digital addition circuits 166 a. Then theadded digital signals that have been stored in the memories 167 a of thehorizontal output circuit 167 for the added signals are outputtedsequentially from the output unit 167A.

And next, according to the timing signal R(2), the first and secondphotoelectrically converted signals of the first and secondphotoelectric conversion units 220 a, 220 b of the plurality of pixels220 of the second row are converted into respective digital signals bythe A/D conversion unit 164, and are sequentially outputted from theoutput unit 165A of the horizontal output circuit 165 for the first andsecond photoelectrically converted signals. Moreover, the first andsecond photoelectrically converted signals that have been A/D convertedby the A/D conversion unit 164 are added together by the digitaladdition circuits 166 a, and the added signals are outputtedsequentially from the output units 167A of the horizontal output circuit167 for the added signals.

Subsequently, sequentially according to the timing signal R(m), thefirst and second photoelectrically converted signals of the first andsecond photoelectric conversion units 220 a, 220 b of the plurality ofpixels 220 of the m-th row are converted into respective digital signalsby the A/D conversion unit 164 and are sequentially outputted from theoutput unit 165A of the horizontal output circuit 165 for the first andsecond photoelectrically converted signals, and the added signals aresequentially outputted from the output unit 167A of the horizontaloutput circuit 167 for the added signals.

Thus it is seen that it will be acceptable for the photoelectricallyconverted signals of the first and second photoelectric conversion units210 a, 210 b of the pixels 210 of the first image sensor 21 and thephotoelectrically converted signals of the first and secondphotoelectric conversion units 220 a, 220 b of the pixels 220 of thesecond image sensor 22 to be added together outside the image sensors21, 22, in other words to be added together by the image generation unit12 b of FIG. 1; or, alternatively, it will be acceptable for thosesignals to be added together in the interiors of the image sensors 21,22, as shown in FIG. 14.

Variant Embodiment #6

In the first embodiment, the first and second photoelectric conversionunits 210 a and 210 b of the pixels 210 of the first image sensor 21 arearranged in the column direction as shown in FIG. 3, in other words inthe vertical direction in FIG. 3A, while the first and secondphotoelectric conversion units 220 a and 220 b of the pixels 220 of thesecond image sensor 22 are arranged in the row direction as shown inFIGS. 3A and 3B, in other words in the left and right direction in FIG.3B. However, it would also be acceptable to provide a structure in whichthe first and second photoelectric conversion units 210 a and 210 b ofthe pixels 210 of the first image sensor 21 are arranged in the rowdirection, while the first and second photoelectric conversion units 220a and 220 b of the pixels 220 of the second image sensor 22 are arrangedin the column direction.

Variant Embodiment #7

In the first embodiment, along with the first and secondphotoelectrically converted signals from the first and second imagesensors 21, 22 being employed as focus detection signals, the samesignals are also employed as image signals. However, for example, itwould also be acceptable to arrange to employ the first and secondphotoelectrically converted signals from one of the first and secondimage sensors 21, 22 as image signals, while employing the first andsecond photoelectrically converted signals from the other of the firstand second image sensors 21, 22 as focus detection signals.

Variant Embodiment #8

FIGS. 15A-15B are figures showing an eighth variant of the firstembodiment. In the first embodiment, as shown in FIGS. 3A and 3B, theshapes of the first and second photoelectric conversion units 210 a, 210b of the pixels 210 of the first image sensor 21 are obtained bydividing the large square pixel shapes equally along the horizontaldirections thereof, while the shapes of the first and secondphotoelectric conversion units 220 a, 220 b of the pixels 220 of thesecond image sensor 22 are obtained by dividing the large square pixelshapes equally along the vertical directions thereof. However, in thiseighth variant embodiment, the shapes of the first and secondphotoelectric conversion units of the first and second image sensor 21,22 are obtained by dividing the large square pixel shapes equally alongthe direction of diagonals thereof. FIG. 15A shows two rows by twocolumns of pixels 210 of the first image sensor 21, in which the firstand second photoelectric conversion units 210 a, 210 b are formed asright angled triangles by dividing the large square pixel shapes alongdiagonals thereof. And FIG. 15B shows two rows by two columns of pixels220 of the second image sensor 22, in which the first and secondphotoelectric conversion units 220 a, 220 b are formed as right angledtriangles by dividing the large square pixel shapes along diagonalsthereof, in a similar manner to the case with the first image sensor 21,but with the diagonal direction of this division being orthogonal to thedirection of diagonal division of the first image sensor 21.

Accordingly, the direction along which the first and secondphotoelectric conversion units 210 a, 210 b of the pixels 210 of thefirst image sensor 21 are arranged, in other words their direction ofarrangement, is different from the direction along which the first andsecond photoelectric conversion units 220 a, 220 b of the pixels 220 ofthe second image sensor 22 are arranged, in other words from theirdirection of arrangement.

Variant Embodiment #9

FIGS. 16A-16B are figures showing a ninth variant of the firstembodiment. In the first embodiment, as shown in FIGS. 3A and 3B, eachof the pixels 210 of the first image sensor 21 and each of the pixels220 of the second image sensor 22 includes first and secondphotoelectric conversion units. However, in this ninth variantembodiment, each of the pixels of the first and second image sensorsincludes first through fourth photoelectric conversion units. In otherwords, in this ninth variant embodiment, the photoelectric conversionunits of each of the pixels are divided into four parts. In FIG. 16A,the pixels 210, 220 of the first and second image sensors 21, 22 haverespective sets of first through fourth photoelectric conversion units210 c through 210F and 220 c through 220 f, each pixel being dividedalong the row direction and along the column direction.

Accordingly if, along with first focus detection signals for the pixels210 of the first image sensor 21 being generated, for example, from thephotoelectrically converted signals of the first photoelectricconversion units 210 c and from the photoelectrically converted signalsof the second photoelectric conversion units 210 d, also second focusdetection signals for the pixels 220 of the second image sensor 22 aregenerated, for example, from the photoelectrically converted signals ofthe first photoelectric conversion units 220 c and from thephotoelectrically converted signals of the third photoelectricconversion units 220 e, then the direction in which the first and secondphotoelectric conversion units 210 c, 210 d of the pixels 210 of thefirst image sensor 21 are arranged, in other words their direction ofarrangement, and the direction in which the first and secondphotoelectric conversion units 220 c, 220 e of the pixels 220 of thesecond image sensor 22 are arranged, in other words their direction ofarrangement, are different.

Moreover, in FIG. 16B, the pixels 210, 220 of the first and second imagesensors 21, 22 have respective sets of first through fourthphotoelectric conversion units 210 c through 210 f and 220 c through 220f, each pixel being divided along a diagonal direction. Accordingly if,along with first focus detection signals for the pixels 210 of the firstimage sensor 21 being generated, for example, from the photoelectricallyconverted signals of the first photoelectric conversion units 210 c andthe photoelectrically converted signals of the fourth photoelectricconversion units 210 f, also second focus detection signals for thepixels 220 of the second image sensor 22 are generated, for example,from the photoelectrically converted signals of the second photoelectricconversion units 220 d and the photoelectrically converted signals ofthe third photoelectric conversion units 220 e, then the direction inwhich the first and fourth photoelectric conversion units 210 c, 210 fof the pixels 210 of the first image sensor 21 are arranged, in otherwords their direction of arrangement, and the direction in which thesecond and third photoelectric conversion units 220 d, 220 e of thepixels 220 of the second image sensor 22 are arranged, in other wordstheir direction of arrangement, are different.

Variant Embodiment #10

In a tenth variant embodiment, an image sensor that employs an organicphotoelectric sheet is built with a two-layered structure.

Due to the “Mg” pixels, the “Cy” pixels, and the “Ye” pixels, the firstimage sensor absorbs for example 50% of the magenta color component, forexample 50% of the cyan color component, and for example 50% of theyellow color component, and passes the remaining portion of the magentacolor component, the remaining portion of the cyan color component, andthe remaining portion of the yellow color component, and also passes thegreen color component, the red color component, and the blue colorcomponent.

In a similar manner to the case with the first image sensor 21, a thirdimage sensor is an image sensor in which organic photoelectric sheetsare used as photoelectric conversion units. This third image sensor isarranged by being laminated to the rear of the first image sensor, and,in a similar manner to the case with the first image sensor, has “Mg”pixels, “Cy” pixels, and “Ye” pixels, and these “Mg” pixels, “Cy”pixels, and “Ye” pixels, along with absorbing the remaining portion ofthe magenta color component, the remaining portion of the cyan colorcomponent, and the remaining portion of the yellow color component thathave been passed through the first image sensor, also passes the greencolor component, the red color component, and the blue color componentthat have been passed by the first image sensor.

The second image sensor is exactly the same as the second image sensorshown in FIGS. 3 and 4, and absorbs and photoelectrically converts thegreen color component, the blue color component, and the red colorcomponent that have passed through the third image sensor.

Accordingly when, for example, the photoelectric conversion units shownin FIG. 15(a) or 15(b) are employed as the first and secondphotoelectric conversion units of the pixels of the third image sensor,then each of the first through third image sensors has a differentdirection in which its first and second photoelectric conversion unitsare arranged in sequence.

Furthermore, it would also be acceptable to provide a fourth imagesensor that employs organic photoelectric sheets and that has “R”, “G”,and “B” pixels between the third image sensor and the second imagesensor. The “R” pixels, the “G” pixels, and the “B” pixels of thisfourth image sensor may absorb for example 50% of the red colorcomponent, for example 50% of the green color component, and for example50% of the blue color component that have passed through the third imagesensor, while passing the remainders thereof. In this way, the first andthird image sensors that have “Mg” pixels, “Cy” pixels, and “Ye” pixelsand the second and fourth image sensors that have “R” pixels, “G”pixels, and “B” pixels are mutually laminated together. By employing astructure such as that of FIGS. 3A-3B, FIGS. 15A-15B, FIGS. 16A-16B orthe like as the structure of the photoelectric conversion units of thesefirst through fourth image sensors, it is possible to make thedirections in which the photoelectric conversion units are arranged insequence be different for each of the image sensors.

Variant Embodiment #11

In the first embodiment the first and second photoelectrically convertedsignals of the first and second photoelectric conversion units 210 a,210 b of a plurality of pixels 210 in the n-th column of the first imagesensor 21 are read out simultaneously to the first and second horizontaloutput circuits 152, 153 respectively, and are outputted sequentiallyfrom the output units 152A, 153A of the first and second horizontaloutput circuits 152, 153. However, in an eleventh variant embodiment,the readout circuit for the photoelectrically converted signals of thefirst and second photoelectric conversion units 210 a and 210 b of thepixels 210 of the first image sensor 21 has the structure shown in FIG.6, in other words has the same structure as the readout circuit for thesecond image sensor 22.

The row scan circuit 161 outputs a timing signal R(m) for signal readoutto the first and second photoelectric conversion units 210 a, 210 b ofthe pluralities of pixels 210. In other words, the row scan circuit 161outputs a timing signal R(1) for signal readout to the first and secondphotoelectric conversion units 210 a, 210 b of the plurality of pixels210 in the first row, and next outputs a timing signal R(2) for signalreadout to the first and second photoelectric conversion units 210 a,210 b of the plurality of pixels 210 in the second row, and thereaftersequentially outputs a timing signal R(m) for signal readout to thefirst and second photoelectric conversion units 210 a, 210 b of theplurality of pixels 210 in the m-th row.

In response to the timing signal R(m), the first horizontal outputcircuit 162 simultaneously reads out the first photoelectricallyconverted signals of the first photoelectric conversion units 210 a ofthe plurality of pixels 210 in the m-th row, and in a similar manner thesecond horizontal output circuit 163 simultaneously reads out the secondphotoelectrically converted signals of the second photoelectricconversion units 210 b of the plurality of pixels 210 in the m-th row.

The first horizontal output circuit 162 outputs from the output unit162A the first photoelectrically converted signals of the firstphotoelectric conversion units 210 a that have thus been read out, andthe second horizontal output circuit 163 outputs from the output unit163A the second photoelectrically converted signals of the secondphotoelectric conversion units 210 b that have thus been read out.

The first photoelectrically converted signals that have been outputtedfrom the first horizontal output circuit 162 and the secondphotoelectrically converted signals that have been outputted from thesecond horizontal output circuit 163 are sent to the focus detectionunit 12 a and to the image generation unit 12 b via the buffer memory 16shown in FIG. 1, and the focus detection unit 12 a performs phasedifference focus detection calculation on the basis of the first andsecond photoelectrically converted signals of the first and secondphotoelectric conversion units 210 a, 210 b of the m-th row that havebeen thus simultaneously read out. Moreover, the image generation unit12 b adds together the first and second photoelectrically convertedsignals of the first and second photoelectric conversion units 210 a,210 b of the pixels 210, and thereby generates an image signal.

In this eleventh variant embodiment, in a similar manner to the casewith the second image sensor 22, the first photoelectrically convertedsignals of the first photoelectric conversion units 210 a of theplurality of pixels 210 that are arranged upon the same row aresimultaneously read out, and also the second photoelectrically convertedsignals of the second photoelectric conversion units 210 b aresimultaneously read out. Due to this, as will be described hereinafter,when an image is obtained by combining the first and secondphotoelectrically converted signals of correspondingly related pixels210, 220 related to the first and second image sensors 21, 22, it ispossible to make the reading out timings of the correspondingly relatedpixels 210, 220 agree with one another, which makes it possible toenhance the quality of the image that is obtained.

Variant Embodiment #12

In the first embodiment the image generation unit 12 b, along withgenerating the first image signal by adding together the first andsecond photoelectrically converted signals of the pixels of the firstimage sensor 21, also generates the second image signal by addingtogether the first and second photoelectrically converted signals of thepixels of the second image sensor 22. However, in a twelfth variantembodiment, the image generation unit 12 b further generates a left eyeimage signal on the basis of, for example, the first photoelectricallyconverted signals of the pixels of the first image sensor 21 outputtedfrom the first horizontal output circuit 152, and also a right eye imagesignal on the basis of, for example, the second photoelectricallyconverted signals of the pixels outputted from the second horizontaloutput circuit 153, thereby generates a first stereoscopic image signal.And, in a similar manner, the image generation unit 12 b also generatesa left eye image signal on the basis of, for example, the firstphotoelectrically converted signals of the pixels of the second imagesensor 22 outputted from the first horizontal output circuit 162, andalso a right eye image signal on the basis of, for example, the secondphotoelectrically converted signals of the pixels outputted from thesecond horizontal output circuit 163, thereby generates a secondstereoscopic image signal.

The image processing unit 14 performs various types of image processingupon the first and second stereoscopic image signals from the imagegeneration unit 12 b, such as interpolation processing, compressionprocessing, white balance processing and so on, and thereby generatesfirst and second stereoscopic image data. This first and secondstereoscopic image data may be displayed upon the monitor 15, and/or maybe stored upon the memory card 17.

Next, reproduction of the first and second stereoscopic image data willbe explained. The first stereoscopic image signal due to the first andsecond photoelectrically converted signals from the first image sensor21 has a parallax in the column direction of the pixels of the imagesensor, in other words in the direction parallel to first and secondpupil regions of the pupil of the photographic optical system 10. In asimilar manner, the second stereoscopic image signal due to the firstand second photoelectrically converted signals from the second imagesensor 22 has a parallax in the row direction of the pixels of the imagesensor, in other words in the direction parallel to third and fourthpupil regions of the pupil of the photographic optical system 10.

Thus, when the face of the observer is upright or erect, the monitor 15shown in FIG. 1 displays a stereoscopic image on the basis of the secondstereoscopic image signal which has a parallax in the row direction; andconversely, when the face of the observer is tilted sideways as forexample when the observer is lying down, the monitor 15 displays astereoscopic image on the basis of the first stereoscopic image signalwhich has a parallax in the column direction.

It should be understood that, in order to change over the stereoscopicimage that is displayed according to the inclination of the face of theobserver in this manner, it will be acceptable, for example, to installan external imaging device to the monitor 15, to capture an image of theface of the observer with this imaging device, to recognize the face ofthe observer with a per se known face recognition unit that is providedto the control unit 12 of FIG. 1, to detect the direction in which theleft and right eyes in the face that has thus been recognized arealigned, and to make the decision as to whether the face of the observeris erect or is horizontally oriented on the basis of this eye alignmentdirection. The changing over of the display of the stereoscopic imageaccording to the inclination of the face of the observer as describedabove may also be implemented upon some monitor other than the monitor15. For example, it would also be possible to transfer the first andsecond stereoscopic image signals to a personal computer or the like,and to change over the stereoscopic image upon the monitor of thispersonal computer between a stereoscopic image based upon the firststereoscopic image signal and a stereoscopic image based upon the secondstereoscopic image signal, according to the angle of inclination of theface of the observer.

Since, as has been described above, according to this variantembodiment, first and second stereoscopic image signals havingparallaxes in mutually different directions are generated, accordinglyit is possible to change over the stereoscopic image display accordingto the angle of inclination of the face of the observer, and due tothis, for example, it becomes possible to provide an effectivestereoscopic view, irrespective of whether the face of the observer iserect or is horizontal.

Embodiment #2

FIGS. 17A-17C are figures showing the fundamental concept of a secondembodiment. While, in the first embodiment described above, with thepixels 210 of the first image sensor 21, the image signal is generatedby adding together the first and second photoelectrically convertedsignals of the first and second photoelectric conversion units 210 a,210 b, here, since as shown in FIG. 17A the first and secondphotoelectric conversion units 210 a, 210 b are arranged with a gap 210c, accordingly the light flux that is incident upon this gap 210 c isnot photoelectrically converted. In other words, in the pixel 210, aneutral zone region 210 c is created in relation to the incident lightflux. In a similar manner, with the pixels 220 of the second imagesensor 22 as well, the image signal is generated by adding together thefirst and second photoelectrically converted signals of the first andsecond photoelectric conversion units 220 a, 220 b, but since as shownin FIG. 17B the first and second photoelectric conversion units 220 a,220 b are arranged with a gap 220 c, accordingly this gap 220 c becomesa neutral zone region in relation to the light flux that is incidentupon the pixel 220. However, in this second embodiment, it is arrangedto reduce the size of this type of neutral zone region related to theimage signals.

As shown in FIG. 17C, when a pixel 210 of the image sensor 21 and apixel 220 of the image sensor 22 that are in a correspondingrelationship are shown as mutually superimposed, the first and secondphotoelectric conversion units 220 a, 220 b of the pixel 220 of thesecond image sensor 22 are present in the greater portion of the neutralzone region 210 c of the pixel 210 of the first image sensor 21, andsimilarly the first and second photoelectric conversion units 210 a, 210b of the pixel 210 of the first image sensor 21 are present in thegreater portion of the neutral zone region 220 c of the pixel 220 of thesecond image sensor 22. Accordingly, as shown in FIG. 17C, the entireneutral zone region of the pixel 210 and the pixel 220 which are in arelationship of correspondence becomes the portion 291 in which theneutral zone region 210 c and the neutral zone region 220 c overlap, inother words becomes the extremely small region shown by hatching.

FIGS. 18A-18D are figures relating to the first and second image sensors21, 22 according to this second embodiment, and shows a method forcombination of the first and second photoelectrically converted signalsof the pixels 210, 220 that are in a relationship of mutualcorrespondence. Here, FIG. 18A shows two rows by two columns of pixelswhich are a portion of the pixels of the first image sensor 21, thesebeing a “Mg” pixel 210, a “Ye” pixel 210, a “Cy” pixel 210, and another“Mg” pixel 210; and FIG. 18C shows two rows by two columns of pixelswhich are a portion of the pixels of the second image sensor 22, thesebeing a “G” pixel 220, a “R” pixel 220, a “B” pixel 220, and another “G”pixel 220, and being respectively in correspondence relationships withthe two rows by two columns of pixels 210 of the first image sensor 21shown in FIG. 18A.

As shown in FIG. 18A, the “Cy” pixel 210, the two “Mg” pixels 210, andthe “Ye” pixel 210 of the first image sensor 21 output CMY imagesignals, and these CMY image signals are converted into RGB imagesignals according to per se known color system conversion processing bythe image processing unit 14 shown in FIG. 1. The RGB image signalsgenerated by this color system conversion processing are regarded,considering the relationship between the RGB image signals and thepixels 210, as though the “Mg” pixels 210 shown in FIG. 18A output a Gsignal, as though the “Ye” pixel 210 outputs a B signal, and as thoughthe “Cy” pixel 210 outputs an R signal. FIG. 18B shows the relationshipbetween the RGB image signals and the pixels 210.

The image processing unit 14 shown in FIG. 1 adds together the imagesignals of the pixels 210 of the first image sensor 21 shown in FIG. 18Band the image signals of the pixels 220 of the second image sensor 22shown in FIG. 18C which are in correspondence relationships therewith.In other words, in FIGS. 18(b) and 18(c), the image processing unit 14adds together the G signals from the upper left pixels 210, 220 togenerate an added G signal, adds together the B signals from the upperright pixels 210, 220 to generate an added B signal, adds together the Rsignals from the lower left pixels 210, 220 to generate an added Rsignal, and adds together the G signals from the lower right pixels 210,220 to generate another added G signal. Of course, instead of addingthese various sets of signals together, it would also be acceptable toaverage them together.

FIG. 18D is a figure showing the pixels 210 of the image sensor 21 andthe pixels 220 of the image sensor 22 that are in correspondencerelationships therewith as mutually superimposed, and schematicallyshows the relationships of the added signals of the overlapped pixels210, 220. The neutral zones related to the added R, and B signals becomethe regions 291 shown by hatching in FIG. 18D, and can be made to beextremely small.

Embodiment #3

The way in which rolling shutter distortion is corrected in a thirdembodiment will now be explained.

FIGS. 19A-19E are figures for explanation of the theory of this thirdembodiment. As described above, the photoelectrically converted signalsof the first and second photoelectric conversion units 210 a, 210 b ofthe pixels 210 of the first image sensor 21 are read out for each columnsequentially in order, while the photoelectrically converted signals ofthe first and second photoelectric conversion units 220 a, 220 b of thepixels 220 of the second image sensor 22 are read out for each rowsequentially in order. Due to this, when the photographic subject ismoving, so called rolling shutter distortion may undesirably take placein the moving image of the photographic subject that is captured.

For example, when the image of a square photographic subject is capturedwhile maintaining the attitude of the digital camera 1 so that the rowdirections of the first and second image sensors 21, 22 remainhorizontal, no distortion takes place in either of the images 181 of thephotographic subject in the images 180 that are obtained by imagecapture by the first and second image sensors 21, 22 as shown in FIG.19A if the photographic subject is stationary. However, if thephotographic subject is moving in the horizontal direction, and if thereadout circuit for the first image sensor 21 is the readout circuitshown in FIG. 5, then the length in the horizontal direction of theimage 181 of the photographic subject that is captured by the firstimage sensor 21 changes according to the direction of moving, as shownin FIG. 19B or FIG. 19C. In other words, when the moving direction ofthe photographic subject is in the direction of the arrow sign shown inFIG. 19B (i.e. the subject is moving in the rightward direction), thenthe image 181 of the photographic subject is expanded; whereas, when themoving direction of the photographic subject is in the direction of thearrow sign shown in FIG. 19C (i.e. the subject is moving in the leftwarddirection), then the image 181 of the photographic subject is shortened.Furthermore, if the readout circuit for the second image sensor 22 isthe readout circuit shown in FIG. 6, then the image 181 of thephotographic subject that is captured by the second image sensor 22 isskewed as shown in FIG. 19D or FIG. 19E, and this is not desirable.

FIG. 20 is a block diagram showing the third embodiment. A first imagesignal acquisition unit 200 sequentially acquires the image signals thatare repeatedly outputted from the first image sensor 21, and a secondimage signal acquisition unit 201 acquires the image signals that arerepeatedly outputted from the second image sensor 22. A moving directiondetection unit 220 detects the direction of moving of a movingphotographic subject on the basis of the image signal from the firstimage signal acquisition unit 200 and the image signal from the secondimage signal acquisition unit 201. This detection of the direction ofmoving can be obtained by comparing together the image signals that arerepeatedly outputted. And the moving direction detection unit 202 couldalso detect the moving direction on the basis of the image signal fromone of the first image signal acquisition unit 200 and the second imagesignal acquisition unit 201.

A selection unit 203 selects the image signal from the first imagesignal acquisition unit 200 or the image signal from the second imagesignal acquisition unit 201 on the basis of the moving direction of thephotographic subject as detected by the moving direction detection unit202. In concrete terms, if the direction of moving of the photographicsubject is the horizontal direction, then the selection unit 203 selectsthe image signal from the first image signal acquisition unit 200, inother words the image signal of the first image sensor 21; whereas, ifthe direction of moving of the photographic subject is the verticaldirection, then it selects the image signal from the second image signalacquisition unit 201, in other words the image signal of the secondimage sensor 22. The image signal that has been selected by theselection unit 203 is either displayed upon the monitor 15 or storedupon the memory card 17. Because of doing this, the image signalselected by the selection unit is not an image having skew distortion,like the images of FIG. 19D or FIG. 19E, but becomes an image signal inwhich there is no skew distortion, like the images of FIG. 19B or FIG.19C.

Furthermore, for example, it would also be acceptable to arrange toprovide a structure which generates an image in which this rollingshutter distortion is corrected by using the image 181 of thephotographic subject shown in FIG. 19B or FIG. 19C, the image 181 of thephotographic subject shown in FIG. 19D or FIG. 19E, and the photographicsubject shifting direction information. In other words, for example,along with detecting an angle a of a specific portion of the image 181of the photographic subject in which no skew distortion such as shown inFIG. 19B or FIG. 19C is present, also an angle 0 is detected of thatspecific portion of the image 181 of the photographic subject in whichskew distortion such as shown in FIG. 19D or FIG. 19E is present. And aphotographic subject image in which no skew distortion is present isgenerated by comparing together these angles α and θ to calculate theangle that originates in skew distortion, and by correcting for thisangle that originates in skew distortion.

It should be understood that the embodiments and/or variant embodimentsdescribed above may also be combined with one another.

While various embodiments and variant embodiments have been explained inthe above description, the present invention is not to be considered asbeing limited by the details thereof. Other aspects that are consideredto come within the scope of the technical concept of the presentinvention are also included within the range of the present invention.

The contents of the disclosure of the following application, upon whichpriority is claimed, are hereby incorporated herein by reference:

Japanese Patent Application 71,017 of 2015 (filed on 31 Mar. 2015).

REFERENCE SIGNS LIST

1: digital camera, 10: photographic optical system, 11: image capturingunit, 12: control unit, 21: first image sensor, 22, 23: second imagesensors, 210, 220: pixels, 210 a: first photoelectric conversion unit,210 b: second photoelectric conversion unit, 220 a: first photoelectricconversion unit, 220 b: second photoelectric conversion unit, 151:column scan circuit, 152: first horizontal output circuit, 153: secondhorizontal output circuit, 161: row scan circuit, 162: first horizontaloutput circuit, 163: second horizontal output circuit, 233: micro lens,233F: focal point, 234: inner lens.

1. An image sensor, comprising: a first photoelectric conversion unitthat generates charge by photoelectrically converting light; a firstelectrode and a second electrode that are arranged in a first directionand output the charge generated by the first photoelectric conversionunit; and a second photoelectric conversion unit and a thirdphotoelectric conversion unit that are arranged in a second directionintersecting the first direction and generate charge by photoelectriallyconverting light that has passed through the first photoelectricconversion unit.
 2. The image sensor according to claim 1, wherein: thefirst photoelectric conversion unit is an organic photoelectric film,and the second photoelectric conversion unit and the third photoelectricconversion unit are arranged on a semiconductor substrate.
 3. The imagesensor according to claim 1, further comprising: a first signal linethat outputs at least one of a signal based upon the charge output bythe first electrode and a signal based upon the charge output by thesecond electrode; and a second signal line that outputs at least one ofa signal based upon the charge generated by the second photoelectricconversion unit and a signal based upon the charge generated by thethird photoelectric conversion unit.
 4. The image sensor according toclaim 3, further comprising: a wiring layer provided between the firstphotoelectric conversion unit and the second photoelectric conversionunit and having the first signal line and the second signal line.
 5. Theimage sensor according to claim 4, further comprising: a lens thatcollects light on the second photoelectric conversion unit and the thirdphotoelectric conversion unit.
 6. The image sensor according to claim 3,further comprising: a wiring layer having the first signal line and thesecond signal line, wherein: the second photoelectric conversion unit isprovided between the first photoelectric conversion unit and the wiringlayer.
 7. The image sensor according to claim 3, further comprising: afirst readout unit that reads out at least one of a signal based uponthe charge output by the first electrode and a signal based upon thecharge output by the second electrode; a second readout unit that readsout at least one of a signal based upon the charge generated by thesecond photoelectric conversion unit and a signal based upon the chargegenerated by the third photoelectric conversion unit; and a wiring layerprovided between (i) a semiconductor substrate having the first readoutunit and the second readout unit and (ii) the first photoelectricconversion unit, the wiring layer having the first signal line and thesecond signal line.
 8. The image sensor according to claim 1, furthercomprising: an addition unit that adds a signal based upon the chargegenerated by the second photoelectric conversion unit and a signal basedupon the charge generated by the third photoelectric conversion unit. 9.The image sensor according to claim 1, which images an image that hasbeen made by an optical system, further comprising: a detection unitthat detects a focal position of the optical system, based upon at leastone of (i) a signal based upon the charge output by the first electrodeand a signal based upon the charge output by the second electrode and(ii) a signal based upon the charge generated by the secondphotoelectric conversion unit and a signal based upon the chargegenerated by the third photoelectric conversion unit.
 10. An imagesensor, comprising: a first photoelectric conversion unit that generatescharge by photoelectrically converting light; a first electrode and asecond electrode that are arranged in a first direction and output thecharge generated by the first photoelectric conversion unit; a secondphotoelectric conversion unit that generates charge by photoelectriallyconverting light that has passed through the first photoelectricconversion unit; and a third electrode and a fourth electrode that areprovided in a second direction intersecting the first direction andoutput the charge generated by the second photoelectric conversion unit.11. The image sensor according to claim 10, wherein: the firstphotoelectric conversion unit and the second photoelectric conversionunit are organic photoelectric films.
 12. The image sensor according toclaim 10, further comprising: a first signal line that outputs at leastone of a signal based upon the charge output by the first electrode anda signal based upon the charge output by the second electrode; and asecond signal line that outputs at least one of a signal based upon thecharge output by the third electrode and a signal based upon the chargeoutput by the fourth electrode.
 13. The image sensor according to claim12, further comprising: a wiring layer having the first signal line andthe second signal line, wherein: the second photoelectric conversionunit is provided between the first photoelectric conversion unit and thewiring layer.
 14. The image sensor according to claim 12, furthercomprising: a first readout unit that reads out at least one of a signalbased upon the charge output by the first electrode and a signal basedupon the charge output by the second electrode; a second readout unitthat reads out at least one of a signal based upon the charge output bythe third electrode and a signal based upon the charge output by thefourth electrode; and a wiring layer provided between (i) asemiconductor substrate having the first readout unit and the secondreadout unit and (ii) the first photoelectric conversion unit, thewiring layer having the first signal line and the second signal line.15. The image sensor according to claim 14, wherein: the secondphotoelectric conversion unit and the wiring layer are provided betweenthe first photoelectric conversion unit and the semiconductor substrate.16. The image sensor according to claim 12, further comprising: a firstreadout unit that reads out at least one of a signal based upon thecharge output by the first electrode and a signal based upon the chargeoutput by the second electrode; a second readout unit that reads out atleast one of a signal based upon the charge output by the thirdelectrode and a signal based upon the charge output by the fourthelectrode; and a wiring layer provided between (i) a semiconductorsubstrate having the first readout unit and the second readout unit and(ii) the second photoelectric conversion unit, the wiring layer havingthe first signal line and the second signal line.
 17. The image sensoraccording to claim 10, further comprising: an addition unit that adds asignal based upon the charge output by the first electrode and a signalbased upon the charge output by the second electrode.
 18. The imagesensor according to claim 10, which images an image that has been madeby an optical system, further comprising: a detection unit that detectsa focal position of the optical system, based upon at least one of (i) asignal based upon the charge output by the first electrode and a signalbased upon the charge output by the second electrode and (ii) a signalbased upon the charge output by the third electrode and a signal basedupon the charge output by the fourth electrode.
 19. An image sensor,comprising: a first photoelectric conversion unit that generates chargeby photoelectrically converting light; a second photoelectric conversionunit that generates charge by photoelectrially converting light that haspassed through the first photoelectric conversion unit; a first readoutunit that reads out a signal based upon the charge generated by thefirst photoelectric conversion unit; a second readout unit that readsout a signal based upon the charge generated by the second photoelectricconversion unit; and a semiconductor substrate having the first readoutunit and the second readout unit, wherein: the second photoelectricconversion unit is arranged between the first photoelectric conversionunit and the semiconductor substrate.
 20. An imaging device, comprising:the image sensor according to claim 19.